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Method and System for Interleaving OCR ^ I Aug. 1999 

and ABL for Automated Mail Sorting 



1. BACKGROUND OF THE invention 



1>1 FIELD OF THE invention 



The present invention relates to automated mail sorting. In 
particular, the present invention relates to method and system 
for increasing reading rates and lowering error rates by 
correlation between optical character recognition and address 
block location techniques* 



1.2 DESCRIPTION AND DISADVANTAGES OF PRIOR ART 



In automated mail sorting systems the ZIP-code of the destination 
address on postmail as e^g. a letter or a parcel has to be found. 
This is usually done by evaluating a digital image of the 
relevant surface of said piece of mail. This evaluation comprises 
firstly a procedure called Address Block Location (ABL) which 
locates a portion of said surface which looks like an address 
block, or has at least some similarity to it. Then, this portion 
is selected and passed to an Optical Recogntion System (OCR) 
system for further processing aimed to extract a ZIP code 
associated with the destination address of the post mail in order 
to be able to sorted according said ZIP code by the sorting 
machine . 

In prior art sorting machines electronic programs running on a 
computer associated with said sorting machine are implemented to 
perform said ABL and said OCR procedures after a digital picture 
was taken of the letter's or parcel's relevant surface area. 

Such a system is disclosed in WO 98 17 4 05. The above mentioned 
procedures are performed sequentially, first ABL, and then OCR. 



With a great portion of letters or parcels such systems work 
quite well, even though there remains an error rate which is 
always worth to be lowered. Errors are very often due to 
confusing the destination address with the return address, both 
written on the surface. 

Problems arise when the size of the envelope or that one of the 
parcel is larger than about DIN B5 as, very often, particularly 
with parcels further remarks or emblems or stamps or the like are 
printed on the surface. This can be misleading for such ABL 
systems because any of such emblem or remarks can be interpreted 
as a potential address block which is then passed to the OCR 
system and will be evaluated in vain as a destination address 
block. 

1 . 3 OBJECTS OF THE invention 

Therefore, an object of the present invention is to provide a 
method and system for increasing reading rates and lowering error 
rates . 

It is another object of the present invention to provide such 
method and system with increased processing speed. 

2. SUMMARY AND ADVANTAGES OF THE invention 

These objects of the invention are achieved by the features 
stated in enclosed independent claims. Further advantageous 
arrangements and embodiments of the invention are set forth in 
the respective subclaims. 

According to the present invention an improved method and system 
for automated sorting machines is provided in which a feedback 
between OCR and ABL takes place. According to a first aspect of 
the present invention a loose coupling between ABL and OCR is 
proposed in which the ABL system is asked for a further address 
block potentially present on the letter's or parcel's surface 
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when the first one being passed from ABL to OCR yielded that no 
destination ZIP code could be extracted from it. Thus, a first 
result of a first OCR procedure is fed back to the ABL system as 
a trigger signal in order to restart the ABL for providing the 
OCR system with a second proposal for a destination address 



Said 'restart ABL' decision is based advantagously on at least 
one of the following types of information: 

The confidence of the actual OCR result, i.e., how good was the 
first 'address block object' readable • Thus, misleading address 
blocks as e.g., emblems having the outer shape of an address 
block, like e.g., a rectangle filled with small symbols or 
characters can be excluded as a sorting criterion which do 
actually not comprise neither any ZIP code nor the name of a 
town. 

Further, as the address block contents are known after being 
processed by the OCR procedure any character information 
recognized by the OCR can be evaluated with some additional 
criteria based on general knowledge. If, e.g., the adress block 
contents comprise a character string like 'Fax', or 'Phone' it 
can be derived that with a high probability the actual address 
block represents the return address instead of the destination 
adress. It is obvious that a lot of further examples for 
knowledge-based criteria can be found* 

Further, ABL confidence is evaluated primarily respecting the 
position, size, outer shape and inner structure found with the 
address block. 

Further, the sort plan actually loaded on the concerned sorting 
machine can be visited for gathering more information. It holds 
information on incoming and outgoing mail, on the town and its 
ZIP code in which the sorting machine is placed. Thus, assuming 
the local town in which the machine is installed is a small town. 



block. 




and the ZIP code evaluated by the first run of the OCR is that 
one of said small town the probability is high that the found 
address block is the return address and must thus be rejected as 
a destination address. The more larger the town, i.e./ the city, 
the less reliable will be such a conclusion as, e.g., in London, 
UK, many letters are destined to another location in London. 

Similarly, postmark information can be evaluated as the stamp 
visible on a postmark designates the name and/or the ZIP code of 
the town in which the letter was stamped, which is in most cases 
the return address town. 

Such evaluations can be performed and qualified in various ways 
independently of the concepts of the present invention. In order 
to concentrate to the real core of these concepts only a simple 
and exemplary way is given next below how to qualify these 
evaluations for a conclusive decision if or if not to accept an 
address block as a destination adress. To do that some qualifying 
results - intermediate or final results - are constructed in form 
of so-called 'confidence values' further referred to as CF-Value 
(i), i 1, 2 4, e.g., which are, e.g., normalized between 0 - 
meaning that no indication can be derived from the evaluated 
information for having found a destination address block - and 1 
- strong indication for having found a destination address. These 
CF-values (i) represent intermediate results related each to some 
special information evaluated. In order to get a usable total 
qualification of an address block all intermediate results must 
be considered, compared to each other and have to be evaluated 
including many particular context conditions which in turn are 
dependent from case to case. Thus, for example by multiplication 
of all CF(i) -values a CF-total value can be yielded. A CF-total 
of 1.0 would then mean that the destination address block is 
found with a probability of nearly 100%. 

With reference back to the restart of the ABL system, said step 
of restarting the ABL system can be of course repeated as long as 
the available processing time allows it until a destination ZIP 




code can be extracted having a high probability of correctness* 

The method of the present invention with the characteristics of 
claim 1 has the advantage, in relation to the method sketched out 
in the discussion of prior art technique that the error rate is 
decreased due to said feedback of information from said OCR 
result to the ABL system. Consequently, the costs for manually 
sorting are reduced. 

In a preferred embodiment of the inventional method as set forth 
in claim 4 a second preferred aspect of the present invention is 
disclosed. Here, the ABL system continues to search further 
address blocks while the OCR system processes one or a plurality 
of address blocks being passed to it from ABL in a preceding 
procedure related to the same letter. After being processed by 
the OCR procedure a summarizing ranking is calculated for each 
potential address block found. The best rank is given for the 
most probable destination address block. In this 'tight' coupling 
of ABL and OCR processes basically the same criteria as mentioned 
above can be used. However, as a further advantage the ABL and 
the OCR processes can be performed concurrently except the first 
ABL process which is the required input for the first run of any 
OCR procedure. A feedback from ABL to OCR or to an 
OCR-and-ABL-embedding 'calling' program part can advantageously 
be provided in order to signalize to the OCR and to said caller 
program part that no more reliable address blocks can be found. 
This represents a really bi-directional feedback between both, 
OCR and ABL which can of course be interleaved as well. Said 
feedback actions can be passed directly from ABL to OCR or, via 
said embedding caller program part from which ABL and OCR 
processes are controlled and invoked, respectively. 

Thus, overall processing time is decreased in relation to the 
work which is done. Further, as all potential address blocks are 
analyzed the error rate can be further reduced. 





3> BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example and is not 
limited by the shape of the figures of the accompanying drawings 
in which: 

Fig. 1 is a schematic representation of a block diagram 

showing the essential steps and the control flow of the 
method according to the first aspect (loose coupling) 
of the invention. 

Fig. 2 is a schematic representation of a block diagram 

showing the essential steps and the control flow of the 
method according to the second aspect (tight coupling) 
of the invention, and 

Fig. 3 is a rough sketch of an envelope front side which is 
subjected to the method according to the first and 
second aspect of the invention. 

4. DESCRIPTION OF THE PREFERRED EMBODIMENT 

With general reference to the figures and with special reference 
now to Fig. 1 the essential steps of the method according to the 
invention are described next below. 

It is assumed that a large size letter is subjected to the 
automated sorting machine having implemented a program performing 
the invent ional method. The letter is destined to an address 
having the ZIP code of Munich, Germany and having a return 
address with the ZIP code of Stuttgart, Germany. Both respective 
address blocks are present on the letter, but, exemplarily, both 
blocks are nearly equal in size and - complicating the case - the 
destination address block is placed at an unusual location at the 
left lower corner of the envelope. Further, the return address 
block comprises the words: 'please notify* and 'FAX 
0711-89898989, z.H., Hr. Maier' . It is assumed further that the 
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postmark information discloses a ZIP code relating to Stuttgart 
as well, and that a remark 

ATTENTION ! 
HANDLE 
WITH 
CARE 

is visible at a location at which usually the destination address 
is found. 

In a first step 110 the letter is subjected to the initial ABL 
procedure/ step 110. As the remark 

ATTENTION ! 
HANDLE 
WITH 
CARE 

has nearly the outer shape of a rectangle and is found on the 
usual location of the destination address block the ABL 
processing will offer that remark first to the OCR procedure, 
steps 110 and 120, Yes branch* The pattern is thus processed by 
OCR. As 100 % of the characters comprising the pattern were 
recognized/ the pure OCR quality is qualified as very well. As, 
however, the remark does not comprise any ZIP code no good 
confidence rating, in form of the above mentioned confidence 
value CF-Value (1), normalized between 0 (no indication for 
having found a destination address block) and 1 (strong 
indication for having found a destination address) , is assigned 
to it, step 130. 

It should be noted that by order of sequence in which they are 
mentioned in the text such CF- values are denoted from CF(1) to - 
CF(4) • Thus, CF(1) would be somewhere around 0,3 . It is obvious 
that the remark cannot yet be excluded as a destination address 
block as it could be that the ZIP code was just forgotten to be 
printed on the letter and the word 'CARE' could be a naime of a 
town as well. 



^^^^^^^^^^^^ 



Then, in a step 140, the confidence of an ABL result relating to 
said current address block just been localized is calculated. 
Thus, a CF(2) value is constructed which reflects e-g,, the 
location of the address block relative to other locations of the 
surface area of the letter. Here, a high value is assigned for 
locations being the usual ones for the placing of the 
destination address block, as e.g*, in the central area of a 
parcel cover, or, the left-central or right-central position on a 
standard DIN B5 letter. Low values, however, are assigned for 
extraordinary locations . 

Further the size and the outer shape and the inner structure of 
the address block found is evaluated by the ABL system or by an 
appended program invoked from a suited location in the program. 
Thus, in the case depicted in fig. 3 a quite high value component 
CF{2)1 is assigned for the location itself as it is quite 
central, further the outer shape ressembling a rectangle having 
straight, horizontal inner line structures yields a high value 
component CF(2)2 as well, and the size being not extra-ordinary 
for a destination address block leading to a quite high value 
component CF(2)3 as well, thus yielding a CF(2) total of around 
0.8 , when the components are aggregated into one resulting 
value. 

Then, in a next step 150, the actually loaded sort plan is 
checked which yields that there would be a high probability for 
Stuttgart to be mentioned in a return address which is however, 
not yet found. As e.g., 'CARE' is the most probable town name in 
the adress block found by the ABL it is seen that CARE does not 
coincide with Stuttgart and it is decided that the current 
address block can still be the destination address block with a 
forgotten ZIP-code. In conclusion, a balanced value of e.g., 0.5 
is assigned as a CF(3) value. 

Then, in a step 160, the postmark information is checked by ABL 
and OCR. It is assumed to reveal Stuttgart and a respective ZIP 
code as town where the letter was stamped. Thus, the information 
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gathered in conjunction with step 150 is further affirmed, and 
the same value of 0*5 is assigned as CF(4) value. 

Then, advantageously, a cross-check of a table comprising towns 
and their respective ZIP code can be performed in order to see, 
if is the name of a town in Germany or within a region in 

Europe, or worldwide, respectively. It is assumed that the result 
is negative, a town 'CARE' is unknown. Also, optionally, a 
cross-check of expressions often found in warnings or further 
remarks written on letters or parcels is performed which would 
of course comprise the text pattern 'handle with care'. 

Then, a comprehensive intermediate conclusion is undertaken, 
primarily respecting the fact if a ZIP code could be extracted 
from the processed address block. It should be vmderstood that, 
generally, there might be respected the results found in steps 
140, 150, 160, as well, for example in cases in which the OCR 
result is quite ambiguous, when e.g., the ZIP code of the town 
has such a large distance to the name of the town that only the 
town name is recognized as a part of the current address block. 
This can then in turn be taken as a trigger for restarting the 
ABL system for locating the missing ZIP-code directly and feeding 
it seperately to the OCR system as an annexe to the address block 
currently being processed. 

Thus, back to the case depicted in fig. 3 and explained in fig, 1 
it is decided in a decision 170 that a further address block is 
asked for from the ABL system without performing a particular 
complex CF-total calculation as depicted in step 180 what will be 
done and described with the next address block, however. Thus, a 
direct feedback step happens from OCR to ABL, and ABL is 
restarted via the NO-branch of step 170. 

Then, in a second run of the ABL the address block depicted in 
the upper left corner of the large size letter which contains the 
return address is proposed by the ABL system and is input to the 
OCR, steps 110, 120. 
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Then steps 130 to 160 are repeated according to the above 
described sequence. Other results are derived, however. In 
particular, performing step 130 is assumed to be well readable, 
and, additionally a ZIP code can be extracted. Further, a text 
string 'PLEASE NOTIFY' and another one, namely, 'FAX' followed by 
a number is extracted from the OCR procedure. As mentioned above 
this is interpreted as an argument indicating that a return 
address is found instead of the destination address. Thus, in 
sxommary, a low CF(1) value of e.g., 0.1 is assigned to said 
second address block indicating that the probability is high that 
the found address is not the destination address. 

Step 140 yields a balanced CF(2) value of around 0.5 is assigned 
as the position is extraordinary, the size, however is normal, 
and the inner structure and outer shape is the expected one of an 
address block. 

Performing step 150 yields a relatively low CF(3) value of e.g., 
0.3, as the sort plan reveals the same ZIP code, i.e., that one 
of the town in which the letter is sorted, as indicated on the 
second address block. 

Then, in step 160 the postmark information is checked yielding 
the same value of CF(4) = 0.3 as the letter was stamped in 
Stuttgart, which is indicated on the actually found address 
block. Thus, steps 150 and 160 yield a high probability that the 
found address block is the return address and not the desired 
destination address . 

Decision 170 is then left via the Yes-branch as at least a ZIP 
code could be extracted. 

Then, in a step 180 a summarizing calculation of the assigned 
CF(i) values is performed for calculating a conclusion if, or if 
not to reject the actually processed address block as a 
destination address. Generally, this calculation can comprise a 
plenty of particular algorithms how to process the CF(i) values. 
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for example a special function which weights the CF(i) values in 
an appropriate way, or some particular direct exclusion in order 
to reject the address block from being used as a destination 
addresS/ as e.g-/ in case of the presence of a FAX number* 
Dependent therefrom the CF(i) values are processed. In here, a 
simple calculation of simply multiplying the calculated CF(i) 
values is proposed for reasons of simplicity only as this 
concerns not the actual core of the invention itself. 

Thus, it is calculated: 0.1 x 0.5 x 0.3 x 0.3 yielding a CF-total 
value of 0.0045. This low total value is quite a good indication 
that the found address block is not the destination adress block. 
Thus, it is concluded in decision 190 to ask the ABL system for 
looking for another address block which is more probably the 
destination address. Thus, a further feedback to ABL takes place 
which involves a third run of ABL which in turn will find the 
destination address in the lower left corner of the letter, 
finally. 

Step 130 yields a high CF(1) value as the address block is quite 
well readable, and a ZIP code, i,e., that one of Munich, Germany 
can be extracted, and no exclusive text strings could be resolved 
from OCR in the address contents. Thus, a CF(1) value of 1.0 can 
be assigned^ 

Further, step 140 is performed which respects the unusual 
position in the upper left corner of the letter's envelope, but 
the good coincidence with an address block concerning size, outer 
shape and horizontal line structure. Thus, a total CF(2) value of 
around 0.7 is assigned calculated by an arithmetic, appropriately 
weighted mean value of the components, for example. 

Further, steps 150 and 160 yield a high CF(3), and CF(4) value of 
both 1.0, as the ZIP code of the sortplan home location and that 
one of the postmark information do not coincide with the ZIP code 
found in the currently processed address block. 
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Thus, decision 170 is left via the YES branch, a summarizing 
CF- total value of 1.0 x 0.7 x 1.0 x 1.0 = 0.7 is calculated in 
step 180 and the conclusion of step 190 is that the CF-total 
value is high enough for proposing a destination ZIP code to the 
sorting machine as, additionally, no direct exclusive conditions 
were found which exclude the address block directly from being 
proposed as a destination address block. Thus, the currently 
found address block is proposed as the destination address block 
of the letter and the ZIP code of Munich is output to the sorting 
machine for sorting the letter into the respective folder 
designated for Munich, step 200, 

Then the sorting procedure has completed, step 210 • 

With additional reference now to Fig. 2 the second aspect of the 
present invention disclosing a tight coupling between ABL and OCR 
processes is described next below. 

Here, the ABL system first processes the remark 'ATTENTION ! 
HANDLE WITH CARE', steps 310, 320, This potential address block 
is passed to the OCR system for further processing, i.e., for 
performing the steps described in conjunction of steps 130, 150 
and 160 of fig. 1, as it is depicted in step 330 of fig. 2. The 
way to process the address block is basically the same as that 
one described above, i.e., CF(i) values are calculated and 
assigned to the address block found. As reveals from step 340, 
however, in contrast to the above described procedure, the CF(i) 
values are collected and saved in dedicated storage area for a 
ranking made up between further address blocks still to be found 
during concurrent processing of the ABL system^ see the second 
YES branch leading to the next looking run of the ABL system. 

Thus, the ABL system continues to search further address blocks - 
here, the next block is that one with the return address - while 
the OCR system may still process the first one. Optionally, a 
buffer memory is provided for storing data the ABL system has 
retrieved and which the OCR system is not yet able to process. 
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As will be appreciated by a person SKilled in the art the 
processes of the ABL and the OCR systems are concurrent in time 
which represents the tight coupling between them and which yields 
a great advantage compared to prior art, i.e*, of ef f iciciently 
using the computational resources present in the automatic 
sorting machines computer system. 

Finally, no more address blocks are found in decision 320. All 
address blocks present on the letter's surface have now been 
located by the ABL system. This can be a decisive difference to 
the embodiment just described above which terminates ABL activity 
when - deviating from the example given in fig. 3 - an address 
block is found which is qualified by the embedding calling 
program as good enough for being accepted as a destination 
address block. In such a case a further address block might be 
found later in time which is considered to have a higher 
probability to be the destination address block than the block 
having the relatively good probability found before. 

After all address blocks were processed by the OCR procedure the 
eiabedding program calculates a summarizing ranking covering each 
potential address block found, step 360. The ranks are compared 
and the best rank is used as an indication for the most probable 
destination address block, the ZIP code of which is output for 
sorting the letter properly, step 370. Then, the sorting 
procedure has completed, step 380. 

In the foregoing specification the invention has been described 
with reference to two specific exemplary embodiments thereof. It 
will, however, be evident that various modifications and changes 
may be made thereto without departing from the broader spirit and 
scope of the invention as set forth in the appended claims. The 
specification and drawings are accordingly to be regarded as 
illustrative rather than in a restrictive sense. 

In particular, the control flow depicted in the drawings should 
be regarded as illustrative only. Many variations are possible to 



achieve the same abstract goals^ i.e., providing a feedback from 
OCR to ABL and/or to run ABL and OCR processes concurrently in 
time . 

Further, a lot of exclusive conditions can be retrieved which 
exclude an address block from being further processed and treated 
as a potential destination address block. Into such 
knowledge-based conditions all general knowledge can be imported 
which has accumulated in the post offices all over the world. 
And/ finally, it will be appreciated that those conditions will 
vary from country to country or from continent to continent. 
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CLAIMS 

1 1 Aug. 1999 

1. A method for automatic mail sorting in which the surface of 
a piece of postmail is scanned with an address block 
locating (ABL) system for locating address blocks which 
after being located are then analyzed by an optical 
character recognition (OCR) system, the method being 
characterized by the step of 

providing a coupling between said address block locating 
system and said optical character recognition system in 
which coupling any results from either of said two systems 
can be fed repeatedly as an input to the respective other 
system for further processing. 

2. The method according to claim 1 in which a loose coupling 
is provided in which said ABL systeia is re-started (110) to 
scan for a further address block if (170) no ZIP code could 
be extracted by said OCR system from the current address 
block currently being ananlyzed. 

3. The method according to the preceding claim in which at 
least one of the following types of information is 
evaluated (130, 140, 150, 160) for a decision to restart 
(110) said ABL system: 

the confidence of the OCR result relating to said current 
address block just being analyzed, 
address block contents information, 

the confidence of the ABL result relating to said current 
address block just been localized, 

data retrievable by a sorr plan associated with the site in 
which the method is run as a program associated with an 
automated sorting machine, 

postmark information retrievable from a postmark, and 
knowledge-based information being derivable from one of or 
a combination of said information. 
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The method according to claim 1 in which a tight coupling 
is provided in which said ABL system continues (320,310) to 
scan for further potential address blocks after having 
found (320) at least one potential address block while said 
address block is processed (130) by said OCR system. 

The method according to the preceding claim in which a 
ranking is provided (340) for each of the address blocks 
being located and/or ananlyzed for finding (360) the most 
probable destination address block, the ranking being based 
on at least one of the following types of information: 

the confidence of the OCR result relating to each of said 
address blocks, 

address block contents information, 

the confidence of the ABL result relating to said current 
address block just been localized, 

data retrievable by a sort plan associated with the site in 
which the method is performed as a program run associated 
with an automated sorting machine, 

postmark information retrievable from a postmark, and 
knowledge-based information being derivable from one of or 
a combination of said types of information. 

The method according to the preceding claim in which 
operation of said ABL system and said OCR system is 
performed at least partly concurrently. 

A system having computerized means for performing the 
method according to one of the claims 1 to 6. 

A program being able to be run on a computer associated 
with an automated sorting machine, said program being 
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provided with code portions representing the method 
according to one of the preceding claims 1 to 6. 



9. 



An automated sorting machine having computer means 



associated with it using the program according to the 
preceding claim. 

10, A computer program product stored on a computer usable 
medium comprising computer readable program means for 
causing a computer to perform the method according to one 
of the claims 1 to 6. 
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LOOK FOR ADDRESS BLOCK (AB) 
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CHECK CONFIDENCE (CP) OF OCR, CALCULATE CF-VALUE (1) 



CHECK CF OF ABL, CALCULATE CF-VALUE (2) 



140 



150 



CHECK CONFIDENCE OF SORTPLAN, CALCULATE CF-VALUE (3) 



T 



160 



CHECK CF OF POSTMARK INFORMATION, CALCULATE CF- VALUE (4) 




FIG. 1 




PERFORM STEPS 13 0 TO 16 0 FROM PIG- 1 
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CALCULATE CF(1), CF(2), CF(3), CF(4) FOR RANKING 
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COMPARE RANKS OF ALL ABs FOUND 



OUTPUT ZIP CODE OF BEST -RANK AB FOR SORTING 
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A method and system for automated sorting machines is provided in 
which a feedback (170^ 190) between OCR and ABL takes place • 
According to a first aspect of the present invention a loose 
coupling between 7VBL and OCR is proposed in which the ABL system 
is asked for a further address block potentially present on the 
letter's surface when the first one being passed from ABL to OCR 
yielded that no destination ZIP code could be extracted from it 
(170) • 

The confidence of the actual OCR and the ABL result, the address 
block contents supplemented with some additional criteria based 
on general knowledge, further, the sort plan actually loaded on 
the concerned sorting machine and the postmark information found 
in the stamps of the postmark can advantageously be taken for 
said ask. 

Further, the ABL system can continue to search further address 
blocks while the OCR system processes one or a plurality of 
address blocks being passed to it from ABL in a preceding 
procedure related to the same letter. After being processed by 
the OCR procedure a summarizing ranking is calculated for each 
potential address block found. The best rank is given for the 
most probable destination address block. In this 'tight' coupling 
of ABL and OCR processes basically the same criteria as mentioned 
above can be used. 
(Fig. 1) 



